Decision Rule#

See the backing repository for Skope Rules here.

Summary#

Decision rules are logical expressions of the form IF ... THEN .... Interpret’s implementation uses a wrapped variant of skope-rules[1], which is a weighted combination of rules extracted from a tree ensemble using L1-regularized optimization over the weights. Rule systems, like single decision trees, can give interpretability at the cost of model performance. These discovered decision rules are often integrated into expert-driven rule-based systems.

How it Works#

The creators of skope-rules have a lucid synopsis of what decision rules are here.

Christoph Molnar’s “Interpretable Machine Learning” e-book [2] has an excellent overview on decision rules that can be found here.

For implementation specific details, see the skope-rules GitHub repository here.

Code Example#

The following code will train an skope-rules classifier for the breast cancer dataset. The visualizations provided will be for both global and local explanations.

from interpret import set_visualize_provider
from interpret.provider import InlineProvider
set_visualize_provider(InlineProvider())

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score

from interpret.glassbox import DecisionListClassifier
from interpret import show

seed = 42
np.random.seed(seed)
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=seed)

dl = DecisionListClassifier(random_state=seed)
dl.fit(X_train, y_train)

auc = roc_auc_score(y_test, dl.predict_proba(X_test)[:, 1])
print("AUC: {:.3f}".format(auc))

/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/sklearn/ensemble/_base.py:156: FutureWarning: `base_estimator` was renamed to `estimator` in version 1.2 and will be removed in 1.4.
  warnings.warn(
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/sklearn/ensemble/_base.py:156: FutureWarning: `base_estimator` was renamed to `estimator` in version 1.2 and will be removed in 1.4.
  warnings.warn(

AUC: 0.918

show(dl.explain_global())

show(dl.explain_local(X_test[:5], y_test[:5]), 0)

Further Resources#

Bibliography#

[1] Florian Gardin, Ronan Gautier, Nicolas Goix, Bibi Ndiaye, and Jean-Mattieu Schertzer. Skope Rules: Machine Learning with Logical Rules in Python. 2017. doi:10.5281/zenodo.4316671.

[2] Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.

API#

DecisionListClassifier#

class interpret.glassbox.DecisionListClassifier(feature_names=None, feature_types=None, **kwargs)#

Decision List Classifier

Currently a slight variant of SkopeRules from skope-rules. scikit-learn-contrib/skope-rules

Initializes class.

Parameters:

  • feature_names – List of feature names.

  • feature_types – List of feature types.

  • **kwargs – Kwargs passed to wrapped SkopeRules at initialization time.

explain_global(name=None)#

Provides global explanation for model.

Parameters:

name – User-defined explanation name.

Returns:

An explanation object.

explain_local(X, y=None, name=None)#

Provides local explanations for provided instances.

Parameters:

  • X – Numpy array for X to explain.

  • y – Numpy vector for y to explain.

  • name – User-defined explanation name.

Returns:

An explanation object.

fit(X, y)#

Fits model to provided instances.

Parameters:

  • X – Numpy array for training instances.

  • y – Numpy array as training labels.

Returns:

Itself.

predict(X)#

Predicts on provided instances.

Parameters:

X – Numpy array for instances.

Returns:

Predicted class label per instance.

predict_proba(X)#

Provides probability estimates on provided instances.

Parameters:

X – Numpy array for instances.

Returns:

Probability estimate of instance for each class.

score(X, y, sample_weight=None)#

Return the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

  • X (array-like of shape (n_samples, n_features)) – Test samples.

  • y (array-like of shape (n_samples,) or (n_samples, n_outputs)) – True labels for X.

  • sample_weight (array-like of shape (n_samples,), default=None) – Sample weights.

Returns:

score – Mean accuracy of self.predict(X) w.r.t. y.

Return type:

float